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Assessing Service Learning in a Communication Curriculum 



As a kind of experiential education, service-learning places the student in the larger civic 
community, usually in a situation where the student engages in work that provides both an 
educational outcome and a benefit to the community. Lillian Stephens’ definition outlines the key 
elements of this pedagogical approach: 

Service-learning is a merger of community service and classroom learning that 
strengthens both and generates a whole greater than the sum of its parts. Service is 
improved by being anchored in the curriculum and learning is deepened by 
utilizing the community as a laboratory for the classroom where students can test 
and apply their curriculum to real-life situations. (1995, p. 10) 

Barbara Jacoby offers a complementary definition: 

Service-learning is a form of experiential education in which students engage in 
activities that address human and community needs together with structured 
opportunities intentionally designed to promote student learning and development. 
Reflection and reciprocity are key concepts of service-learning. (1996, p. 5) 

The service-learning model incorporates classroom experience, community experience, and 

reflection that applies classroom theory to community involvement and community reality to 

classroom lessons. 

Programs of communication study have increasingly embraced service learning. David 
Droge and Bren Murphy (1999) provide both a comprehensive overview of programs utilizing 
service-learning and almost a dozen examples of service-learning’s application to specific 
communication courses. Despite the growing excitement about service-learning for 
communication and the many reports of increased student learning, few have systematically 
explored whether it accomplishes its full potential. Droge and Murphy include only one essay 
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that deals with assessing the service-learning project (Bergstrom & Bullis, 1999) and the 
annotated bibliography that concludes the volume (Fisher, Wechsler, & Kendall, 1999) mentions 
only four items that deal explicitly with assessment. 

Though assessment activities run through all teaching (in the form of examinations or 
projects-all those things that allow teachers to assign grades to students), an assessment of 
service-learning should go beyond the basic final grade in a class. In fact, a systematic 
assessment of service-learning should incorporate lessons from the educational assessment 
literature as well as from the service-learning literature 

Educational Assessment 

Assessment combines teaching and research in that it systematically collects and analyzes 
data to test the hypothesis that students have met pedagogical goals. At the same time it goes 
beyond this due to the complex nature of the educational process, which finds itself pressed by 
political, economic, and educational forces, each with its own set of goals (Erwin, 1991, pp. 2-3). 
Educational assessment must examine not only student learning but also program success. Two 
strands of definition emerge: (A) “Assessment is the process of defining, selecting, designing, 
collecting, analyzing, interpreting, and using information to increase students’ learning and 
development” (Erwin, 1991, p. 15). (B) “Three purposes appear most frequently in definitions of 
evaluation: (1) to render judgments on the worth of a program; (2) to assist decision-makers 
responsible for deciding policy; and (3) to serve a political function” (Talmadge, 1982, p. 594, 
quoted in Worthen & Sanders, 1987, p. 24). The second strand picks up the economic and 
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political factors, while the first highlights the purely educational one. Both strands appear in 
service-learning programs. 

Assessment makes the most sense when individuals or institutions can outline clear 
educational goals and outcomes. These targets could include mastering a body of knowledge, 
facts, or concepts; learning how to think critically, understand ideas, or apply concepts; 
producing academic or artistic products; grasping procedural skills; and developing dispositions 
and interests (Brookhart, 1999, pp. 10-12). Teachers and administrators may make a further 
distinction, following Michael Scriven, between formative and summative assessment. The 
former provides ongoing feedback during the course of a program while the latter occurs at the 
end, to provide a judgment about a program’s overall merit (Worthen & Sanders, 1987, p. 34). 

All of these things clearly relate to service-learning in that it ambitions multiple goals for at least 
three constituencies: students, community agencies, and schools. Because assessment best suited 
to measure student outcomes may not help community groups decide whether to participate in a 
service-learning project, for example, teachers and program administrators should develop 
several assessment tools. 

The assessment literature suggests that objectives-oriented evaluation or assessment best 
meets such diverse needs. This involves establishing broad objectives, classifying and defining 
them in terms of behavioral outcomes, finding situations in which achievement of these goals can 
be shown, and measuring them through appropriate techniques (Worthen & Sanders, 1987, p. 

63). Thomas Angelo and K. Patricia Cross suggest that a cluster or Q-sort analysis works well to 
focus the goals; they further suggest that educators limit assessment activity to between five and 
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eight clusters of outcomes (1993, p. 16). Once program participants establish the goals, then they 
can design methods to test them. The measurement of the accomplishment of these goals can 
take many forms, most of them already known to teachers: criterion- or norm- or standardized- 
referenced tests, demonstrations, essays, experiments, group projects, performances, portfolios, 
and projects (Fischer & King, 1995, pp. 3-4). 

Though not explicitly envisioning service-learning assessment, T. Dary Erwin offers 
some important advice for successfully examining educational outcomes that applies to service- 
learning. Assessment must involve many people: faculty, administrators, students, off-campus 
participants, and any other constituent groups (1991, pp. 24-25). Student cooperation is essential, 
particularly in obtaining their understanding the purposes of the assessment and their willingness 
to take it seriously, not just for their own grade, but for the long-term benefits of the program (pp. 
28-29). Finally, Erwin notes that assessment results do most good when all the constituent groups 
have access to them and can discuss them. 

A successful assessment of service-learning should begin with clearly specified goals, 
classified in related clusters, and agreed upon by the constituent groups. Each cluster should have 
clearly identified evaluation measures. Finally evaluation results should be shared and discussed 
at least among faculty, students, community groups, and service-learning administrators. 

Goals for Service-Learning 

Most service-learning programs accept a common set of outcome objectives. These 
include such student-focused goals as enhancing student learning through practical experience, 
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balancing theory and practice, fostering a sense of caring for others, using skills, promoting 
growth in civic values, gaining a sense of diversity and multiculturalism, assisting growth in 
personal relations, building solidarity with underserved groups, learning from the disadvantaged 
their perspective on the attitudes and values of the dominant society, fostering moral 
development, and aiding spiritual growth. Other, community-based goals include empowering 
community groups as places of learning, providing new energy to community services, enabling 
more services from existing agencies, connecting to new resources, and obtaining more helpers. 
For colleges and universities, the goals comprise improved student learning, holistic education 
(“the education of the whole person”), insertion into the civic community, promoting diversity 
and multiculturalism, and the promotion of social justice (Soukup, 1996). 

Communication programs may specify one or more of these. “Using skills” may become 
speaking before community groups, editing a newsletter, or developing a web site. “Balancing 
theory and practice” may encompass participant observation in a homeless shelter in order to 
discover group norms and group formation. “Assisting growth in personal relations” could 
involve addressing issues of communication apprehension through dialogue with multicultural 
groups. However expressed, service-learning program goals demand clear articulation. 

The goals outlined above can be sorted or categorized into fewer broad outcome 
objectives. The following example of this process draws on the service-learning program at my 
own institution. The interpretation of the goals-particularly the emphasis on social justice, 
solidarity with the underserved members of the community, and the spiritual and moral 
development of the students-reflects the service-learning program at Santa Clara University; 



Assessing Service Learning 
page 6 



other schools may omit or add some or other components. Component parts of the categories 
appear in parentheses after each major goal. 

1. Intellectual development of students 

(Enhance understanding of the subject matter, bridge theory and practice, 
understand other cultures, understand civic problems and goals) 

2. Skills development in students 

(Develop writing, speaking, and other forms of expression; practice classroom- 
taught techniques; experience “real-world” situations) 

3. Affective development in students 

(Develop civic values, grow in personal relations, increase empathy, establish 
multicultural connections) 

4. Moral and spiritual growth of students 

(Connect with others, grow in solidarity with underserved groups, understand 
social justice concerns, see others as peers) 

5. Community outcomes 

(Increase numbers of volunteers, provide new services, tap into new resources, 
develop long-term connections by teaching students) 

6. College or University outcomes 

(Forge connections with community, participate in local partnerships, develop 
social outreach, test new educational model) 



Assessing the Goals-A Test Case Study 

To examine the feasibility of assessing service learning along these lines, this study 
undertook a secondary analysis of data collected by Santa Clara University’s Eastside Project 
from 1990 to 1999. The data, originally collected as part of routine student feedback, consists of 
narrative evaluations from 1990 to 1993 and again from 1995 to 1999; survey evaluations from 
1991 to 1993 and from 1995 to 1997. 

Participants 

For the purposes of this analysis, this study reduced the data set, originally collected from 
across the university, to those students who participated in service-learning placements in 
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conjunction with their enrollment in communication courses. Since students voluntarily filled in 
the various instruments, the data set does not include all students who participated in the service- 
learning components of their courses. Another limitation emerges from the fact that the Project 
staff used different instruments in the earlier and later periods. 

Students enrolled in communication courses between 1990 and 1999 (N = 577) 
completed the narrative evaluations at the end of the academic quarter in which they participated. 
Similarly, in some terms from 1992, 1993, 1995, 1996, and 1997, students (N = 382) also 
completed survey forms. The university collected no data in 1994. 

Instrumentation: Narrative evaluations 

Placement options varied, with most communication students working at homeless 
shelters, day-care centers, tutoring programs, nursing homes, and community centers. The 1990- 
1993 narrative evaluations (N = 88) consisted of 17 items (see Appendix I for the full set of 
questions). After demographic questions, the form requested input on why the student chose the 
placement, the student’s role at the placement, surprises in the service-learning project, 
connections to academic work, understanding of social issues, and suggestions for improving the 
experience. 

The 1995-1997 narrative evaluations (N = 489) used a shorter form (see Appendix 13) that 
students completed at the placement site rather than in the classroom. After the demographic 
items, the assessment requested information about first impressions, learning, feelings about the 
people, and connections with course content. 
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Instrumentation: Survey 

The Eastside Project staff used several survey forms during the nine years of data 
collection. In order to document the kinds of assessment undertaken, I will mention all of the 
surveys, though only two will be analyzed. The first survey form consisted of 32 items, with nine 
demographic items and 23 Likert-type scales, mostly focused on social attitudes (see Appendix 
m). This survey had a pre-test, post-test design. (An unknown number of communication 
students completed this form; the codebook for the survey was accidentally discarded and. data 
identifiers cannot be reliably reconstructed.) The second survey form consisted of 25 items, four 
demographic questions plus 21 Likert-type scale items, seeking student opinion about the 
placement, service-learning, social issues, and the people at the placement (see Appendix IV). 
Forty-five students enrolled in communication courses completed this survey. Finally, the 1995- 
1997 survey had 31 items, 14 demographic and 17 Likert-items on similar topics to those in the 
second survey (see Appendix V). The greatest number of students enrolled in communication 
courses completed this form (N = 357). 

A factor analysis of the 1992-1993 data (using a principal component analysis extraction 
and a Varimax rotation with Kaiser normalization) yielded five factors: the value of the 
experience and its influence on students’ future behavior, the initial experience of the placement, 
the connection with academics, the value of discussion groups, and value of contact with clients 
(see Table 1 for the components and the loadings). 
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Table 1: 1992-1993 survey 


Components 


1 


2 


3 


4 


5 ’ 


31. Overall I found this a worthwhile experience 


.836 










30. 1 would recommend placement to others in class 


.821 










29. 1 would return to this placement 


.796. 










28. As result of this, I am likely to volunteer in future 


.740 










21. My curiosity engaged & I want to learn more 


.730 










32. It has deepened my understanding of social issues 


.624 










27. 1 feel more confident interact with others of 
different backgrounds 


.510 










15. The staff was helpful 




.849 








12. Orientation helped reduce my anxiety 




.810 








13. 1 felt welcomed at this placement 




.716 








11. Orientation helped my understanding of my role 




.713 








14. 1 would have liked more direction at placement 




-.574 








22. Placement was enhanced by classroom learning 






.912 






19. Placement enhanced my understanding of subject 






.874 






18. Understood how experience fits educational goals 






.722 






20. This influenced my research and class assignment 


' 




.579 






25. Liked discussion groups on subject matter of class 








.825 




24. Like discussion groups on improving experience 








.781 




23. Discussion groups helped me connect with course 








.470 




17. Placement provides sufficient opportunity to 
interact with clients 










.701 


16. 1 thought tasks at this placement worthwhile 










.698 
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A factor analysis of the 1995-1997 data (using a principal component analysis extraction 
and a Varimax rotation with Kaiser normalization) yielded four factors: the value of the 
experience and its influence on students’ future behavior, the connection with academics, the 
awareness of community and social issues, and the initial experience of the placement (see Table 
2 for the components and the loadings). 



Table 2: 1995-1997 survey 


Component 


1 


2 


3 


4 


23. 1 would recommend this placement 


.801 








25. 1 found this a worthwhile experience 


.789 








22. 1 would return to this placement 


.785 








13. 1 thought the tasks I did were worthwhile 


.740 








14. Placement provides sufficient opportunity to interact with 
clients 


.689 








1 1. 1 felt welcomed at the placement 


.679 








12. The staff was helpful 


.675 








16. This experience enhanced my understanding of subject 
matter of the course 




.836 






15. 1 understood how it fit with goals of class 




.772 






17. Experience influenced my research 




.646 






19. Placement was enhanced by classroom 




.578 






18. Curiosity was engaged to learn more 




.567 






20. Feel more confident interacting with those of different 
backgrounds 






.866 




24. Deepened my understanding of social justice issues 






.612 






12 



Assessing Service Learning 
page 11 



21. More likely to volunteer in future 






.530 




10. Orientation reduced my anxieties 








.934 


9. Orientation helped me understand my role 








.923 



Instrumentation: Other 

In addition to these forms, service-learning participants completed departmental narrative 
evaluations and, in many cases, kept a journal of their participation. Sadly, the departmental 
forms did not collect data about the Eastside Project, and faculty returned the student journals at 
the end of the term without keeping any record beyond the grade. From all this, only some 
anecdotal material remains. 

The Eastside Project staff also compiled another questionnaire that the placement 
supervisors completed each term. This does not appear in the data set. 

Results 

Narrative evaluations 

The data sets provide information about a variety of topics relevant to the assessment 
clusters outlined above. An initial reading of the student narrative data shows that the responses 
highlight five themes: connections with learning, student personal issues, empathy and relating to 
others, changes in outlook, and program feedback. 

Many of the students did make connections with their course material, though in general 
terms. One student wrote, “[I learned] how the surroundings and media influence the way 
children grow up and interact with their environment” (34F97). Another commented, “I think 
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what we have been discussing in class has enhanced/covered what I have been doing. For 
example, non-verbal communication, hand gestures, etc.” (35F97). Others managed to connect 
the experience to specific skills. “I have applied the placement more towards developing my 
communication skills” (13S98ii). Yet another felt good about “the fact that I relate a lot of what I 
am doing through non-verbal and verbal communication. It makes me more aware of what 
messages I send and how that affects her answers back to me” (16S98). 

Sometimes the learning takes a different direction from the class material, but still fulfills 
the goal of student development. Two journalism students commented on this. “I got to do a lot 
of one-on-one reporting that was very useful for class (writing news articles)” (50F96). Or “I was 
able to learn more about homelessness through my class assignments (i.e., interviewing people 
for stories, etc.)” (55F96). An introductory student who had worked in a senior center applied 
lessons from both interpersonal and mass communication: “I learned the importance of clear 
verbal and non-verbal communication. Also, I learned that even though media tends [sic] to 
picture elderly to be physically frail (which is true), they are not as mentally “frail” as the media 
depicts [sic]. They are quite on the ball” (40W95B). 

But the placements did not automatically lead to learning outcomes. A good 
number-perhaps 10% to 15%-simply noted that they did not see any value in the service- 
learning experience. Or, if they saw a value, it was in the volunteer work, but not in any way 
connected to class. A typical comment of this type takes this form: “Actually, I don't feel any 
significant connection between the two. When I’m writing in my journal, I feel like I have to find 
some kind of connection” (18F97). Others were even more blunt. “There was not any academic 
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knowledge that I could make a direct connection with my experience” (1 19A9093). And again, “I 
didn't feel that this experience really tied into any of my academics at SCU” (125A9093). 

The narrative data indicates that most of the students do make connections to their 
learning, but at different levels. However, the data do not indicate why the experience works for 
some but not for others. 

Second, another theme in the responses reflects personal issues of the students. For some, 
their service-learning placement continued a process begun in high school service projects. For 
many others, this became their first insertion into a local community, especially a local 
community with needs. A good number of narrative responses, then, expressed satisfaction, a 
feeling of personal value or accomplishment, or a sense of emotional attachment. Some noted 
that the most important part of their experience was learning about themselves. Typical 
comments include statements like these: “I am very proud of my involvement at the Almaden 
Center. I feel I made a positive contribution to the Center” (26S95). “I learned to accept and 
better understand how homeless men feel and think. Also by interacting with the men I was able 
to obtain an entirely different perspective on life, but one that is simpler and more applicable!” 
(10W95B). Sometimes this personal growth focused on one or other aspect of the student’s life: 
“I learned how to relate to people who differed in culture and social background from me” 
(17S95). Other times, students made a personal judgment based on the site. One, who did not 
care for the elder care placement, wrote, “I learned that I shall never put my parents in a 
retirement home” (18F96). 

Overall, students seemed most aware of personal issues as they completed the narrative 
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evaluations. One benefit from the service-learning experience is in the area of emotional growth: 
a sense of self-esteem, a sense of being appreciated, a sense of satisfaction. 

A third theme in the narrative evaluations grows from the emotional. Students reported a 
growth in empathy for those they served. This cuts across the kinds of placement as the following 
comments show. A student working at a nursing home wrote, “I know I learned it because I value 
what I have more now, and have a deeper sense of empathy for those at the convalescent 
hospital” (45W95B). One tutoring at an inner-city school commented, “I learned that studying 
hard was not the only requirement for success in school. I came from a background that gave me 
an advantage. Most of the people in the program had far more obstacles to graduating from high 
school than I ever did” (22W95B). Another student working with the homeless wrote, “I learned 
that the homeless are real people. I learned that sometimes things that aren't planned happen 
(7A9093). Finally, someone in a community center teaching English as a second language 
reflected, “I better understand the attitudes and perspectives of the Hispanic person towards me 
and our society!” (12A9093). 

This notion of relating to others runs through many of the responses. For a good number 
of program participants, the key idea of service learning seemed to be their ability to relate to 
others, particularly those different from themselves. This shows up in two, apparently different 
ways, seeing others as the same or seeing them as different, as the next two comments indicate: 
“The kids’ lives and experiences were so different from mine. I enjoyed talking to them and 
hearing about their lives” (54F96). “They're just like you and me. They're friendly and intelligent 
too” (55F96). For many of the upper-middle class students at Santa Clara, this similarity to others 
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from different backgrounds came as a huge revelation. 

Fourth, a theme of changed outlook appears in the data. This could apply either to one’s 
outlook on others, similar to what we saw in the third theme, or to one’s outlook on education or 
life. Here are representative comments, first dealing with judgments about people in the 
community centers: “My thoughts have completely changed, these kids are good and they have 
aspirations, they just need someone to help them meet those goals” (22S98ii). Second, about the 
educational enterprise: “Learning isn’t all books and classes. We can learn through experiences 
and observations” (8A9093). The service-learning experience does work to change students. 

Finally, a number of students seriously address the question about changing or improving 
the program. For many, this became a more difficult item to answer, since it required accepting 
the invitation to criticize the authority figures of the placement sites and of the University staff. 
One wrote about a tutoring site, “I was just amazed at the lack of respect, the vulgar language 
that the kids displayed. The staff didn't help much. All of us were in complete disgust! I would 
not recommend this site to anyone” (4S97). Another noted, “80%- 90% of the time I did office 
work. I don't mind helping out wherever needed, but for the purpose of this Eastside, it was not a 
worthwhile time commitment” (25S97). Perhaps not surprisingly, most simply noted that nothing 
should be changed. 

The narrative data also indicates spots of program failure. In addition to the explicit 
suggestions students make, some directly note that they did not learn from the experience (as 
illustrated above). More, though, indirectly indicate some lack of learning. A telling indicator 
that runs through most of the 1990-1993 comments and continues to a lesser degree in the 1995- 
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1998 comments is that the students refer to themselves as “volunteers” and see their service- 
learning activities as “volunteer work.” In a program that tries to separate itself from University- 
sponsored volunteer work, that bills itself to faculty, students, and administrators as an 
educational component supplementary to the classroom, this failure to explain itself is a serious 
issue. Another failure flag comes in the many student comments whose language reflects a sense 
of separation from the community through a kind of magnanimity, a “we will be good enough to 
help you” kind of mentality. The kinds of learning from the community envisioned by service- 
learning planners has not yet caught on with these students. 

Survey data 

In the 1992-1993 data set (N = 45), an analysis of means shows that the students tended 
to agree or strongly agree with almost all the questions; in fact the rates tend to be in the 70% to 
90% range. The only item that falls below this is item 22 “What I learned at my placement was 
enhanced by classroom activities and assigned readings.” Only 57% agreed with this statement, 
while 42% either disagree or strongly disagreed. 

Of the factors, two seem most relevant to the assessment clusters identified above: the 
value of the experience and its influence on students’ future behavior (factor 1) and the 
connection with student academics (factor 3). There were no significant differences between 
subsets of these, based on year in school or on numbers of service-learning experiences that the 
students had. (Unfortunately, the gender variable was corrupted, so no comparisons could be 
made on that basis.) However, the levels of general agreement with the items constituting factor 
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1 differed significantly from the levels of agreement with the items constituting factor 3. While 
levels of agreement were both positive, students agreed with factor 1 items 70% to 90% of the 
time and with factor 3 items 50% to 65% of the time. (The average mean of items in Factor 1 = 
1.602; average mean of items in Factor 3 = 1.968; t = -3.231, df = 31; p c.003). 

The 1995-1997 data set (N = 357) shows similar pattens, with students manifesting the 
same tendency to agree or strongly agree with most of the items, usually at the 80% to 90% rate. 
This set exhibits the same weaknesses as the earlier one: the items loading on factor 2, the 
connection with academics, consistently show lower levels of agreement (though these are still 
mostly above 50%). Item 16 “This experience enhanced my understanding of the subject” 
showed 63% agreement and 35% disagreement. Item 17 “This experience influenced the research 
andd/or other assignments I did for this class” had 63% agreement and 32% disagreement. Item 

18 “My curiosity was engaged enough by this experience that I wanted to learn more about the 
issues and people related to this placement” scored 67% agreement and 31% disagreement. Item 

19 “What I learned at my placement was enhanced by classroom activities and assigned 
readings” showed the lowest levels of agreement at 48%, with 47% disagreeing. Here, too, the 
levels of agreement with each factor differed significantly. (N=336. The average mean of items 
loading on Factor 1 = 1.858, average mean of items loading on Factor 2 = 2.266, df = 335, p < 
. 001 ). 

On the two key factors, the value of the experience and its influence on students’ future 
behavior (factor 1) and the connection with academics (factor 2), there were no significant 
differences between subsets based on numbers of service-learning placements. However, males 
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and females did differ significantly on making connections with academic work, with the women 
(N = 258) more likely to agree than the men (N = 88) with the statements loading on factor 2 
(male mean = 12.25; female mean = 11.135; t=2.274, df= 123.46, p = .025). The students also 
differed significantly on their agreement with those items loading on factor 1 (worth of the 
experience) according to year in school. Here the sophomores showed less agreement with the 
rest (oneway analysis of variance: freshmen N=78, mean = 12.3, sophomores N= 137, mean = 
13.92; juniors N = 67, mean = 12.25; seniors N = 55, mean = 12.85. F=2.753, df = 3 (between), 
333 (within), p. < .05). 

Other measures 

Anecdotal data from the student journals, based on an informal polling of faculty 
recollections, indicates that between 50% and 66% of the students do “get it,” that is, do make an 
application of their service-learning experience to their communication course material. The 
more successful results come in applied courses such as journalism and reporting classes or 
documentary video. In each of these, the students gather data for course assignments at their 
placements, interviewing people and writing a feature story, for example. As one journalism 
teacher put it, “Even if the students hate the experience, they learn a lot of reporting skills that 
will serve them well when they leave school.” The more general courses like the introduction to 
communication study or the interpersonal course show less clear-cut results. Those teachers who 
stress the connections and urge the students to regard their service-learning experience as a kind 
of participant observation report that the students do grow in the ability to recognize and record 
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communication behaviors. Similarly, the intercultural communication course also reports that 
students benefit from experiencing the variety of cultures in the Northern California milieu of the 
University. 

Discussion 

The results of the data analysis indicate several important things about assessing the 
service-learning component of the communication courses at Santa Clara University. The 
program is broadly successful, but not perfect. Four key weaknesses need addressing. 

First, and perhaps most important, we cannot adequately assess the experience because 
we lack comparison data. While the Communication Department, the Eastside Project office, and 
individual faculty collected data, they did so independently and no group collected data on 
students who do NOT participate in service-learning activities. In the classes that use service- 
learning, it is required of all students, so faculty members cannot compare grades or other 
measures of student learning. Similarly, departmental course evaluations do not compare across 
classes; as they are currently implemented, too many other variables would make comparisons 
impossible. (For example, summer session sections do not use service-learning placements, and 
therefore could provide a comparison group, but summer session courses differ so dramatically 
from those scheduled in the regular term-number of class meetings, meeting times, class length, 
accelerated scheduling, etc.— that direct comparison cannot be made.) Eastside evaluations only 
collect data from student participants; there is nothing from those not participating. 

What we can assess indicates at least one weakness in the program’s implementation in 
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the communication curriculum at Santa Clara. Both narrative and survey data indicate that 
students do not make connection to their class material; they fail in the last step of service- 
learning: classroom reflection on the experience. Somehow the students do not make the full 
circle from theory (classroom) to practice (placement) to integration (reflection on practice). 
Faculty implementation of the service-learning model needs improvement. 

Second the kinds of data collected do not address each of the six clusters of objectives of 
the service-learning experience: intellectual development of students, skills development of 
students, affective development of students, moral and spiritual growth of students, community 
outcomes, university outcomes. The narrative data yield five general themes that line up only 
partially with the clusters: (1) connection with learning, (2) personal issues, (3) empathy, (4) 
changes in outlook, and (5) program feedback. The survey data yield four factors that similarly 
align themselves only partially: (1) the value of the experience and its influence on students’ 
future behavior, (2) connection with academics, (3) awareness of community and social issues, 
and (4) the initial experience at the placement. 

What we have seen breaks down in this fashion. 



Objectives cluster 


measures 


Student intellectual development 


faculty grading reports 
student journals 

student self-report (Eastside survey, factor 2 or 3) 
Eastside narrative evaluations, theme 1 


Student skills development 


faculty grading reports 


Student affective development 


student self-report (Eastside narrative evaluations: 
themes 3 and 2) 
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Student moral, spiritual development 


student self-report (nothing directly addresses this, 
but it may emerge indirectly) 


Community outcomes 


Not addressed in the current evaluations, except 
indirectly in the Eastside narratives, theme four. The 
supervisor’s report may touch on some issues. 


University outcomes 


Not directly addressed in any evaluation instrument, 
but may appear indirectly in narrative comment, 
particularly in themes 2 and 4, and in factors 1 and 3 
of the survey instrument. 



The greatest value of this line up lies in its ability to indicate what we do not know. 
Faculty should explore ways to measure whether a service-learning placement helps in skills 
development and in affective development. At some point the University should investigate 
whether service-learning placements help its interests in the local community and the local 
community should ask whether its members benefit from the university commitment. At the 
moment we have only anecdotal evidence, as when a placement supervisor complains about the 
time it takes to brief students each term, questioning whether the “costs” to the placement match 
the “benefits” from the students’ participation. 

Third, the way that the data has been collected makes long-term analysis difficult. Both 
narrative and survey instruments have changed over the years. In some years no survey data was 
collected; in others, question wording changed enough to render questionable any combination of 
data sets. 

Fourth, students have also noted that they feel over-evaluated. In some courses they 
complete five or six different evaluation instruments, in addition to the normal academic 
evaluation tools of papers, journals, and exams. Because faculty and program administrators have 
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not clearly explained the process to them, students have not really seen the value of the 
assessment and participate somewhat half-heartedly, as many of the narrative evaluations show. 
At the same time, students constitute the core of the service-learning assessment. In this, the 
University has missed out on the participation of other key players, especially the faculty, 
program administrators, and off-campus participants other than the supervisors. To my 
knowledge, no one has asked people served in the various placements what they think of the 
students and the students’ work. Despite the claim that the students work at their placements to 
learn from the community, those “teachers” do not participate in evaluating student work or the 
University’s role in the community. 

Finally, we should not lose sight of the fact that the service-learning program has a good 
proportion of successes. Students do evaluate the program positively and self-report that they 
benefit. Students do show an increased interest in and commitment to the local community. The 
University does break down town-gown barriers. Faculty show support for the program. 

We learn two things from assessment: what we should look for in student outcomes and 
what we should look for in program planning. The more we learn about each, the more we 
succeed in both. 



ERIC 



24 



Assessing Service Learning 
page 23 



References 



Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college 
teachers. (2 nd Ed.). San Francisco: Jossey-Bass Publishers. 

Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of 

pedagogy. ASHE-ERIC Higher Education Report (Vo. 27, No. 1). Washington, DC: The 
George Washington University, Graduate School of Education and Human Development. 

Droge, D., & Murphy, B. O. (Eds.). (1999). Voices of strong democracy: Concepts and models 
for service-learning in communication studies. Washington, DC: American Association 
for Higher Education. 

Erwin, T. D. (1991). Assessing student learning and development: A guide to principles, goals, 
and methods of determining college outcomes. San Francisco: Jossey-Bass Publishers. 

Fischer, C. F., & King, R. M. (1995). Authentic assessment: A guide to implementation. 
Thousand Oaks, CA: Corwin Press, Inc. 

Jacoby, B. (1996). Service-learning in today’s higher education. In B. Jacoby (Ed.), Service- 
learning in higher education: Concepts and practices (pp. 3-25). San Francisco: Jossey- 
Bass. 

Soukup, P. A. (1996). Inviting others to take the helm: Service-learning and the marginated 
community. Paper presented at the 82 nd annual meeting of the Speech Communication 
Association, San Diego, CA. November 1996. 

Stephens, L. S. (1995). The complete guide to learning through community service, grades K-9. 
Boston: Allyn and Bacon. 

Worthen, B. R., & Sanders, J. R. (1987). Educational evaluation: Alternative approaches and 
practical guidelines. New York: Longman. 




25 



Assessing Service Learning 
page 24 



Appendix I 



Eastside Project Student Evaluation Form 
[1990-1993, spacing adjusted for this reproduction] 

Your participation in community placements through the Eastside Project should be a positive 
one both educationally and personally. In addition, this experience should enhance your 
understanding of the community which your placement serves. 

Please complete this form to let us know about your experience this quarter. Your observations 
and comments will be used to improve the work of the Eastside Project in facilitating community 
placements for Santa Clara students. 

YOUR NAME: COURSE: 

YEAR: PLACEMENT: 

1) Why did you choose this placement? 

2) How much time did you spend at the placement? 

Each week: Throughout the quarter: 

3) How did you spend your time at this placement? List all tasks or responsibilities. 

4) What percent of your time was spent in working directly with the people who this placement 
serves? 

5) What surprised you about this placement? 

The people: 

The setting: 

The work: 

6) What was the most significant aspect of your work at this placement? 

7) What did you learn from the people who you met at this placement? 

8) What connections were you able to draw between your experience at this placement and your 
academic experience? 

9) How has your understanding of the social issues changed because of your experience at this 
placement? 
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10) What helped you or could have helped you to prepare for your volunteer experience? 

11) If you were to participate in this placement again, what would you do differently? 

12) What could this placement do to improve students’ experience? 

13) Please comment on the Eastside Project’s role in your placement. 

Suggestions: 




27 



Assessing Service Learning 
page 26 



Appendix II 



Placement-Based Assessment 
Eastside Project Thoughts 

[1995-1997, spacing adjusted for this reproduction] 

Name SS# 

Placement Date 

This is my (first, second, third, etc) Eastside Project placement. 

1. (a) When I arrived at this placement, my first impressions were: 

(b) When I first came here, I felt the least prepared to deal with: 

2. So far, as an active learner at this site, I have learned that: 

3. After spending some time at this placement, my beliefs/thoughts/feelings about the people I 
am meeting here are changing in these ways: 

4. I’m beginning to make some new connections between the content of my course 

(name one course for which you are doing this placement), and what I’m noticing at this site, for 
example: 

and that makes me want to have more class discussions (or write my course paper) about: 



Thank you for completing this assessment. 

Please take this opportunity to talk with your placement supervisor about your ideas. If you have 
issues about your Eastside Project learning that are not covered on this form, please contact the 
Eastside office immediately at 554-4549. 

White copy: Faculty Yellow copy: Eastside Project Pink copy: Placement 
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Appendix HI 

Santa Clara University Eastside Project 

[1993 evaluation form] 

1 Gender: A. Female B. Male 

2. Ethnic background : A. African - American B. Asian/Pacific Islander C. Caucasian 

D. Hispanic E. Native American A and B. Other 

3. Year in School: A. Frosh B. Sophomore C. Junior D. Senior E. Other 

4. Including this quarter, how many quarters have you worked with the Eastside Project or SCAAP? 

A. One B. Two C. Three D. Four E. Five or more 

Please use the following key in responding to the following questions: 

A = Strongly Agree B = Agree C = Disagree D = Strongly Disagree 

5. It is the moral obligation for every person to protect the welfare of persons beyond his/her circle of 
family and friends. 

6. An individual or small group of people can have an impact on a community problem. 

7. Opportunities to provide service exist at all times. 

8. Skills and experiences that I gain from service learning will be valuable in my career. 

9. Service learning will help me develop leadership skills. 

10. I worry that my skills may not be adequate in working at my placement. 

11. I have some concerns for my personal safety while at my placement. 

12. Service learning will challenge me to interact with diverse groups of people. 

13. I have the ability to communicate with people of different races and backgrounds. 

14. I am inspired by service learning to make a greater contribution to society. 

15. Anyone who tries hard enough can get ahead in this world. 

16. A university does not have a responsibility to encourage its students to combat social injustice. 
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17. The value of service learning is overestimated by most people. 

18. Diversity is critical to community development; differences must be respected and sought while 
similarities are explored and developed. 

19. My knowledge of social problems and issues is expanded through service learning work. 

20. Most people on welfare are trying to beat the system. 

21. I can learn a great deal about society when I take the perspective of the poor and homeless. 

22. There are important lessons about human living which can only be learned through direct contact 

with the poor and powerless members of our society. 

23. My experience with service learning will effect my choice of majors and/or careers. 

24. The measure of a society's greatness rests on how the poor and vulnerable members are treated. 

25. Most poor and homeless people could get a job if they tried. 
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Appendix IV 

Santa Clara University 
Eastside Project 
Program Evaluation 

[1992-1993 evaluation form] 

General information: Answer the following questions beginning with space number 

1. This evaluation is for which quarter? 1. Fall 2. Winter 3. Spring 

2. Your class: l.Freshman 2. Sophomore 3. Junior 4. Senior 

3. Gender: 1. Male 2. Female 

4. For this class, participation in an Eastside Project placement was: 1. Required 2. Optional 

5. How many Eastside Project placements have you completed? 

l.One 2. Two 3. Three 4. Four 5. More than four 

6.1 attended the orientation for this placement before I began my work. 1. Yes 2. No 

7. Which option most closely reflects the amount of time per visit you spent at your placement? 

1. Less than 60 mins. 2.60 90 mins. 3.90 mins. 2 hrs. 4. more than 2 hrs. 

8. Which option most closely reflects the number of weeks you attended your placement? 

1. Less than four 2. Four 3. Five to six 4. Seven to eight 5. More than eight 

9. Which option most closely reflects the percent of time per visit you spent working directly with the people 
whom this agency serves. 

1. less than 25% 2.25% - 49% 3.50% - 64% 4.65% - 79% 5. 80% - 100% 

Please use the following key in responding to questions 10 - 32 below: 

1 = Strongly Agree 2 = Agree 3 = Disagree 4 = Strongly Disagree 5 = Not Applicable 

10. During sign-ups I was able to select a placement recommended by my instructor. 

The orientation was helpful in: 
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11. developing my understanding of my role at the placement. 

12. reducing my anxiety about my first visit to the placement 

13.1 felt welcomed at this placement. 

14.1 would have liked more direction and supervision at my placement. 

15.The staff at this placement was helpful. 

16.1 thought the tasks I was asked to do at this agency were worthwhile. 

17.This placement provides sufficient opportunity for students to interact and become acquainted with the 
clients whom the agency serves. 

18.1 had a clear understanding about how this experience fit into the educational goals of this course. 

19. This experience enhanced my understanding of the subject matter for this course. 

20. This experience influenced the research and/or other assignments I did for this class. 

21 .My curiosity was engaged enough by this experience that I wanted to learn more about the issues and 
people related to this placement. 

22. What I learned at my placement was enhanced by classroom activities and assigned readings. 

23 . Discussion groups sponsored by the Eastside Project helped me to connect course content with my 
placement experience. 

I liked it when these discussion groups focused on: 

24. how to improve the placement experience. 

25. subject matter specific to the course. 

26. I would have liked to have discussion groups with people who were all working with the same 
population, e.g., children or elderly. 

27 As a result of this experience, I feel more confident interacting with people whose race or ethnic and 
cultural background is different from my own. 

28. As a result of this experience, I am more likely to volunteer in the community in the future. 

29. Given the opportunity, I would return to this placement. 
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30. I would recommend this placement to other students who take this class. 

31. Overall I found this to be a worthwhile experience. 

32. This experience has significantly deepened my understanding of social issues and social justice. 
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Appendix V 

Santa Clara University 
Eastside Project 
Program Evaluation 

[1995-1997 evaluation form] 

General information: Answer the following questions beginning with space number 
This evaluation is for which quarter? l.Fall 2. Winter 3. Spring 
Your major: 

1. Your class: l.Freshman 2. Sophomore 3. Junior 4. Senior 

2. Gender: 1. Male 2. Female 

3. For this class, participation in an Eastside Project placement was: 1. Required 2. Optional 

4. How many Eastside Project placements have you completed? 

1. One 2. Two 3. Three 4. Four 5. More than four 

5. How well did the experience at your placement meet your expectations? 1. Exceeded 5. Failed 

6. Which option most closely reflects the amount of time per visit you spent at your placement? 

1. Less than 60 mins. 2.60 90 mins. 3.90 mins. 2 hrs. 4. more than 2 hrs. 

7. Which option most closely reflects the percent of time per visit you spent working directly with the people 
whom this agency serves. 

1. less than 25% 2.25% - 49% 3.50% - 64% 4 . 65 % - 79% 5. 80% - 100% 

8. How was the level of supervision? 

1. Way Too much 2. Too much 3. About right 4. Needed more 5. Too little 

Please use the following key in responding to questions 10 - 32 below: 

1 = Strongly Agree 2 = Agree 3 = Disagree 4 = Strongly Disagree 5 = Not Applicable 

The orientation was helpful in: 

9. developing my understanding of my role at the placement. 
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10. reducing my anxiety about my first visit to the placement 

11. I felt welcomed at this placement. 

12. The staff at this placement was helpful. 

13. 1 thought the tasks I was asked to do at this agency were worthwhile. 

14. This placement provides sufficient opportunity for students to interact and become acquainted with the 
clients whom the agency serves. 

15. 1 had a clear understanding about how this experience fit into the educational goals of this course. 

16. This experience enhanced my understanding of the subject matter for this course. 

17. This experience influenced the research and/or other assignments I did for this class. 

18. My curiosity was engaged enough by this experience that I wanted to learn more about the issues and 
people related to this placement. 

19. What I learned at my placement was enhanced by classroom activities and assigned readings. 

20. As a result of this experience, I feel more confident interacting with people whose race or ethnic and 
cultural background is different from my own. 

21. As a result of this experience, I am more likely to volunteer in the community in the future. 

22. Given the opportunity, I would return to this placement. 

23. I would recommend this placement to other students who take this class. 

24. This experience has significantly deepened my understanding of social issues and social justice. 

25. Overall I found this to be a worthwhile experience. 
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